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ABSTRACT 


This thesis outlines the use of the Training- Modeling Integration TM-! process for 
development of training information. High resolution simulations portray weapon sys- 
tem operations in sufficient detail for the training developer to use the simulation to 
formulate task information and training insights. Training developers have relied heavily 
on observable information for developing training. Throught the use of the TM-] 
process, training developers can now use high resolution simulations to observe system 
employment and operation. Straightforward data analysis techniques are used to ana- 
lyze simulation derived data files. The unique nature of this methodology is embodied 
in the synthesis of training development needs, analytical techniques and high resolution 


simulation data. 
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I. INTRODUCTION 


High resolution combat simulations have been used extensively in the military for 
over 20 years. Their primary use has been to gain insight into the character and nature 
of the major elements in a battle. The use of high resolution models have been confined 
almost exclusively to hardware evaluation and force structure analvses. Through the use 
of these simulations the military analyst 1s able to examine weapon system and force 
structure performance. Using approved measures of effectiveness the analyst makes 
comparisons between various Weapon systems or force structures and their contribution 
to the combat power of the force. These combat simulations are the cornerstone of all 
weapon system and force structure evaluations. 

There is another critical function that these high resolution simulations can play: 
the derivation of training development information. Examining any of the production 
high resolution simulation models used by the Army reveals there 1s much more 
information generated than results related only to weapon and force structure 
performance. One area of interest that spans all branches and functions within the 
Army 1s training information. Training information is obtainable from high resolution 
models and can aid significantlv in training program development. 

Currently TRADOC Analvsis Command (TRAC) at White Sands Missile Range, 
New Mexico is using high resolution model output to develop realistic target arravs for 
the 7th Army Training Command. Through characterization of the events that take 
place within a series of similar simulation runs the analysts at White Sands have 
characterized tank engagements on the battlefield. Through this characterization thev 
can describe likely target arravs and target array components. This report addresses an 
extension of this tvpe of analvsis by consideration of specific tasks performed by weapon 
system crews. Gaining insights into tasks, conditions, and standards as performed in a 
simulation provides valuable information for intergration into weapon system training 
programs. 

The nature of production high resolution simulations allow for gathering specific 
task information throughout the battle. Through careful selection of specific task data 
the trainer can build characterization of tasks much in the same way White Sands has 
built characterizations of the battlefield environment. The significance of this type of 


analysis might be overlooked if 1t Were not for the fact that critical training information 


can now be obtained years in advance of current training development techniques. 
Heretofore the training developer was not able to observe operational performance of 
new svstems until field testing of prototype systems. Production simulation runs for new 
systems are available vears prior to field testing . In the past the training developer had 
to wait until the svstem became fully operational to see it employed in a combat 
environment. Through the use of high resolution simulations the training developer can 
observe the svstem in simulated combat and gain valuable information vears prior to 
prototype testing and evaluation. 

Although the Army has used high resolution models for many vears it is hard to 
explain why the training developer has not taken advantage of the information generated 
by high resolution simulations. Admunistratively the trainer and combat developer have 
had separate organizations causing some problems in easy exchange of information. in 
particular emerging ideas related to uses of simulation outputs. The trainer has 
historically relied on observed data or subject matter expert (SME) data for development 
of training programs. Simulation derived data is neither observable nor viewed as 
Operational data. Although careful examunation of current production high resolution 
simulations reveals extremelv high levels of fidelity, the training developer has habitually 
relied on SMEs and questionaire data gathering techniques. 

The nature of current high resolution simulations allow for the recording of 
numerous events. historv files, and their time of execution within the simulation run. 
History files allow the analyst to record specific actions or series of actions taken by the 
model throughout the course of the battle. Consideration of the simulation, not as a 
model, but actual combat allows the training developer to record and monitor the 
svstem’s performance in combat from a training developer's perspective. By running a 
number of these battles the training developer gains insights into the character of the 
tasks, the conditions under which all the tasks were performed and time standards that 
were present in the model. If the service is using simulation data for force structure 
decisions and Weapon system procurement decisions, it 1s logical to have the training 
developer use the conditions and standards displayed in the simulation for training 
development. 

The training development information 1s very dependent upon the the accuracy and 
fidelitv of the model. Since current models are not written to address training 
development issues the representation of critical training development items may not 


always be present. Because of the close interrelationship of svstem employment and 


training, most training development data items are represented to some degree in good 
high resolution simulations. As fidelity increases and models improve there will be a 
similar increase in the degree of training development data obtainable from high 


resolution simulations. 


Il. METHODOLOGY AND DATA SETS 


A. METHODOLOGY 

The Training-Modeling Integration (T-MI) process developed at TRAC, White 
Sands will be used to extract the training development information from the high 
resolution model. The augmented T-MI process used for this report provides specific 
training development information for a new helicopter system, the AH-64, based upon 
the performance of the system in a high resolution force-on-force combat simulation. 
The analvsis of these force-on-force battles creates an artifical experience base used by 
the training developer to characterize and define the employment of the new svstem. 
This experience base now allows the training developer to define specific parameters: 
range, number of targets viewed. target behavior, hkely target combinations and 
engagement time lines. This type of specific information about task performance 
provides the training developer with the essential building blocks of the training 
program: TASKS, CONDITIONS, and STANDARDS. 

A brief explanation of the importance of tasks. conditions and standards is needed 
to set the stage for their development within this thesis. Training development products 
include manv elements; critical task lists. terminal learning objectives. lesson plans and 
periods of instruction (POI) are just a few of the major products. Tasks are just one of 
these major element and are further defined by specific conditions and standards. 
Specific conditions and standards for weapon system tasks form the basis of the training 
program’s POI and establishes the framework for all training related activities. The 
following definitions further define these critical elements within the training 
development process. 

e TASK--An event or activity that has a defineable starting point and is measurable. 


¢ CONDITION--The situation or environment in which a soldier or unit 1s expected 
to accomplish a task in actual practice. 


¢ STANDARD--A description of the performance which a unit or an individual must 
meet in order to demonstrate minimum acceptable performance in a task. 
Standards are based on the level of performance required for mussion 
accomplishment or battlefield survival. 
Table 1 depicts the flow of information and the analvsis activities conducted to 
formulate training development information from the T-MI process. The process starts 


with the selection of a high resolution model and an approved scenario emploving the 


weapon svstem of interest. The simulation model is executed 20 to 40 times to produce 
sufficient replications of the history file to support the statistical analysis. This history 
file contains the event history of the battle that was fought in the simulation. In most 
cases the content of the history file must be decided very early in the analysis in order 
to insure critical activities are in fact being recorded in the history file. The 
postprocessors of production models will have to be programmed to capture needed 
event data. This recording of events or activities constitutes the indirect observations 
of the battlefield by the training developer. For the purposes of this report existing 
AH-64 history file information was used. To assist the T-MI process, an aggregation 
of related events and activities was conducted within the postprocessor to assist in the 
interpretation of the data. 

Concurrent with the running of the simulations, the TRADOC proponent school 1s 
developing the initial training task lists for the new weapon system. These task lists 
describe the individual and collective tasks to be performed by the operators of the new 
svstem. Additionally, the SMEs on the new svstem are formulating their personal 
concepts on conditions of employment and performance standards. It is critical to 
formulate this reference base, task lists and SME data, for the conditions and standards; 
otherwise, the simulation history files are hard to interpret and relate to battle events. 

Initial integration of the aggregated simulation runs, history files, and the initial task 
list is the first major activity of the T-MI Team. The T-MI1 team examines the 
simulation output for data items that relate to specific task performance. This analvsis 
extracts from the historv file the data base items that characterize the system events 
during the conduct of battle. These records of events, task histories, form the 
observations for the svstem operator, SMEs and the training developer. 

Task history analysis 1s accomplished through a statistical analvsis on the type, 
frequency, duration and character of events associated with each task. The result 1s a 
confident description of the characteristics of each task, (1.e., conditions and standards). 
Not all tasks listed on the individual and collective task lists will be modeled or 
characterized in the simulation; however, those critical point target Weapon svstem 
related tasks, (i.e., main gun firing, , laser designation, missile launch constraints, etc.), 
will be represented and will provide the T-MI Team with data to formulate training 
development information. The character of the events within task histories and the 
resulting statistical analvsis is highly dependent upon the scenario chosen for the 


simulation model. A variety of scenarios available in the high resolution simulation 


should be run to broaden the data base for the training developer and the analyst. For 
institutional training base programs, multiple scenario analysis will be necessary to 
insure general applicability of the tasks, conditions and standards derived from the T-MI 


analysis. 


Table 1. T-MI TRAINING DEVELOPMENT METHODOLOGY 


Step A. High resolution simulation scenarios are developed for the 
new system, multiple scenarios desired 
Bl. 40 replication history file | B2. Svstem critical task list 
is prepared developed 


Step A. T-MI Team reviews history file and initial task list 
B. Task characterizations are identified by T-M1 Team 


Step Conduct analvsis of history file data to develop 
characterizations for selected critical tasks 


Step 4 A. T-MI Team reviews statistics from the data analvsis 
B. New conditions and standards are developed for initial tasks 





B. DATASETS 

The data sets needed for the analysis are obtained from two separate agencies within 
TRADOC. Each set of data is critical for an effective analvsis, and the analvsis can not 
be performed if either set of data has not been fully and properly developed. A suitable 
high resolution production model playing the particular weapon system of interest is the 
cornerstone of the analysis. All statistical analvsis is conducted on the history files from 
this high resolution model. The information obtained from the training developer is not 
in the form of a data file; rather it is a listing of the critical tasks essential to system 
Operation and employment. The front end analysis is performed by the training 
developer to obtain the initial task list. Appendix A outlines the steps taken within a 
training development front end analysis (FEA). Simulation data files and the task list 
share equal importance in the analysis. The task lst, combined with SME expertise, 
focuses the statistical analysis. Without proper focusing, the results of the T-Ml 


analvsis would be unrelated to the specific training requirements of the weapon svstem. 


Through the use of these two data sets the analvst and training developer are able to 
relate simulated battlefield observations to real world training requirements. 
1. Simulation Data 

As mentioned earlier, the high resolution simulation data will be stored in a 
history file. The training development analyses will most likely have to use existing 
history files from runs of production models used for force development and weapon 
system acquisition purposes. Within the Army the model that has been used extensively 
for the last ten years is CARMONETTE. Although no longer in favor for current 
analyses, CARMONETTE 1s appropriate for use in training development analyses for 
fielded weapon systems. The newer high resolution model, CASTFOREM, does not 
have sufficient scenarios developed that play currently fielded svstems. For emerging 
Or proposed weapon system training development, CASTFOREM will be used. The 
data for this report was drawn from a CARMONETTE history file emploving the 
AH-64 Apache helicopter. The scenario played in the simulation was an approved and 
validated scenario, used for the AH-64 and other studies. The training developer 1s 
concerned that the scenario has been approved and validated in order for the results to 
be integrated into the total training program for the weapon svstem and force. 
Appendix B is an outline of this specific scenario. From the T-MI analyst’s perspective 
the critical element of this particular scenario is that it plavs the weapon system of 
interest, AH-64, in sufficient quantity to obtain many observations of typical battlefield 
actions and events. This history file contains over 8800 events and actions concerning 
the AH-64. These 8800 events were generated by running the same CARMONETTE 
scenario 40 times and recording the occurrence of the same event elements. 

2. Training Data 
Training data is in the form of an initial task list which is an output of the FEA, 


the preliminary step in the development phase of the System Approach to Training 


(SAT). The initial task list forms the basis for the entire planning and programming of 
the weapon system's training program. In the case of the AH-64 the training program 
included purchase of several computer driven training devices, motion simulators, an 
eight week institutional training program, use for two different aircraft for training and 
individual aviator career management. Appendix C contains the mission tasks from the 


initial task list used for early training development work on the AH-64. 


lil. ANALYSIS 


This section explains and discusses the steps outlined in the methodology. The most 
critical aspect of the analysis process is the synthesis of the needs of the training 
developer with model information. The training developer’s primary task is relating 
available simulation data to tasks, conditions and standards of the weapon system's 
training program. The analvst’s task is to interpret the available simulation data for the 
training developer. Maintaining focus on the desired training development products will 
be the goal of all T-MI Team members. 

The methodology describes each step of the analysis using representative samples. 
The samples used for illustrative purposes are reflective of the more difficult analytic 
efforts and provide insight into the underlving analytical processes and techniques used 
in simulation characterizations. Five tasks from the many listed on the initial task list 
are developed. These tasks reflect critical battlefield events and are the fundamental 
training requirements for the svstem operators. A thorough understanding of battlefield 
events that involve these tasks adds significantly to the training developer's ability to 
develop a meaningful operator training program. The number of actual tasks reflected 
in the high resolution model is limited by the degree of resolution in the model. As 
resolution increases in production simulations, more of the initial tasks will be reflected 
within the model. 

Activities required in the methodology are grouped into steps based on logical 
associations. As mentioned earlier, many of the activities are conducted simultaneously 


by agencies with other uses for the developed information. 


A. SpE! 


Table 2 lists the activity blocks in the methodology that comprise step one. 


Table 2. STEP 1 ACTIVITIES 


A. High resolution simulation scenarios are developed for the 
new system, multiple scenarios desired 





Bl. 40 replication history file | B2. System critical task list 
is prepared developed 


This step is characterized by each major party, the modeling community and the training 


community, developing their resepective initial products separately. The modelers 
develop appropriate scenarios for a high resolution model for the new weapon svstem 
and the trainers develop the initial task lst for training. These two actions are currently 
required for all major weapon svstems and are conducted by TRADOC proponent 
school organizations. At present these two activities are conducted with little interface 
between the two organizational elements. 

Selecting a suitable simulation model with the appropriate resolution and scenarios 
is the first major effort in the analysis process. Early in most acquisition cycles scenario 
selection will be limited, but an effort must be made to gain a broad spectrum of 
scenarios in order to generalize the results of the training development products. The 
CARMONETTE model was chosen because it is the only high resolution inodel with 
approved scenarios and accessable history files for the AH-64 attack helicopter. Only 
One scenario is analyzed in this thesis, but others need processing and their results 
aggregated into the current analysis. The second critical element of Step 1 is 
development of the initial or critical task list for the weapon system. This document 1s 
produced by the proponent school for the weapon svstem and 1s developed using the 
System Approach to Training (SAT). Appendix A briefly outlines the steps and 


processes to develop the initial task list. This initial task list is critical to the T-M] 
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process because the entire analysis effort will be devoted to furthering the training 
developer’s understanding of these tasks. Focusing on these critical tasks is a unique 
aspect of using the T-MI process for training development. The critical task list used in 
this analvsis was obtained from the training developer at the L.... Army Aviation Center 
and is the original task list developed in 1981 for the AH-64. The mission task listing 


of the task list is contained in Appendix C. 


B. STEP 2 

Step two is the initial effort by the T-MI Team to integrate training requirements 
and model information. Efforts in this step are focused on understanding the 
components of the tasks and the abilitv of the model to reflect or characterize anv of 
these components. Task components are the conditions and standards associated with 
the particular task. The T-MI Team meets to determine the degree to which each task 
on the critical task list can be characterized within the model. Although the high 
resolution model portrays many events and actions of the weapon system, not all critical 
tasks are represented in the simulation. In this step the team identifies those tasks that 
are likely candidates for characterization and development. The team will not truly 
know if the characterizations will materialize until the data analysis begins. Candidate 
tasks must be selected to narrow the scope of the analysis and focus the analvst’s efforts. 
Since the task list is long and certain tasks are much more critical than others, the 
training developer will prioritize those tasks requiring simulation data analysis and 
eliminate those tasks that can be more easily developed using other training development 
techniques. The tasks listed in Table 3 are the tasks that will be analyzed. 

Part of the training developer’s responsibility will be to educate the analvts on 
unique terminology. In the case of the AH-64 there are many unique terms and systems 
associated with the weapon system. Understanding the items described in this brief 


listing will aid in this analysis. 


1] 


e Hellfire Modualar Missile Svstem (HMMS)--The Hellfire missile is the primary 
armorment svstem on the AH-64. The missile is laser guided and mounts on the 
winglets of the AH-64. The copilot gunner is the primarv crewmember to employ 
the fea levio: 


¢ 30mm Chain Gun--This gun is mounted under the forward section of the aircraft 
and can be fired bv both crewmembers. The gun is fully articulated and is used 
against secondary targets at close range. 


e Lock On Before Launch (LOBL)--This firing mode is one of the two possible firing 
modes of the HMMS. In this mode the laser seeker in the missile sees the target 
before the missile departs the AH-64. 


¢ Lock On After Launch (LOAL)--This firing mode is employed when the laser 
seeker does not see the laser designation prior to launch and acugqires the target in 
flight. 


e Autonomous Designation--This designation technique is accomplished when the 
firing AH-64 laser designates his own targets. This method of designation has a 
higher probability of hit due to single aircraft use. 


¢ Remote Designation--Remote designation is characterized by other than the firing 
Alj-64 designating the target. Possible designators include other AH-64s, OH-58D 
aircraft and field artillery forward observers. 


Table 3. TASKS FOR ANALYSIS 


Task #6066 Search and Identify Targets 
Task #6080 Engage Targets with Iellfire Missile 


Task #6082 Engage Multiple Tgts ~w Two Weapons simultaneously 
Task #6404 Perform LOBL Autonomous Designation Engage Procedures 
Task #64505 Perform LOBL Remote Designation Engage Procedures 





Selection of tasks that are represented within the model defines the scope of the 
analytical effort for the analyst. Through a careful review of the historv files, specific 
data points are related to initiation, termination, frequency or other aspects of the task 
that the training developer feels will aid in his development of the conditions and 
standards. It is essential that the analyst understands what the developer needs from the 
characterization or the analysis will be unusable by the training developer. For the 


tasks selected above the essential characterizations needed are outlined in Table 4. 


Table 4. TASK CHARACTERIZATIONS 


Frequency of shots while in battle 
Percent of shots taken in remote status 
Number of targets viewed through weapon system (possible tgts) 
Total shots fired in thirty minutes of battle time 


Percent of hits from all shots taken 
Number of multiple hits on same targets (overkills) 
Expected survival time while in battle 
Range to targets when hit 





In addition to being keyed to the task list, the characterizations listed in Table 4 
reflect several critical aspects of any battle: pace of the battle, survival function of the 
AH-64 and the rearm or reload requirement that is inherent to all weapon systems. 
Developing a training program based upon expected performance levels of the svstem 
and critical aspects of the battle will allow the trainer to emulate realistic battlefield 


conditions in the training environment. 


Cr oTEP 3 

Step three initiates the statistical analvsis of the model output. The statistical 
principles are straightforward and not difficult to implement. The most difficult task 1s 
organizing the history file data and establishing the data in a easily used statistical 
format. The CARMONETTE simulation run data was postprocessed within the VAX 
mainframe at White Sands. This postprocessing collated all the engagement sequences 
between Red and Blue elements during each run of the simulation. For this processing 
only the events involving Blue helicopter elements were collated. Forty replications of 
the simulation were compiled to form the data base. The data base was delivered in 
ASC}]] text file format and downloaded onto the NPS IBM mainframe computer. The 
APL SCRUB function presented in Appendix D reads the ASCII file and creates a 
usable CMS data matrix. Once in a usable form the data was manipulated and 
interpreted using the APL functions listed in Appendix E. The APL functions produce 
vectors and statistics that describe the essential information associated with each task. 
GRAFSTAT 1s used exclusively to analyze the data. GRAFSTAT was selected because 
of its thorough statistical analvsis package and the ease with which the graphs can be 
displaved within this text. Other statistical packages possess similar analysis capability 


and would serve equally well. 


NN) 


The analysis utilizes the following basic principles; 


¢ Each of the characterizations listed in Table 4 is a random variable and insight into 
the behavior of these characterizations 1s made through development of a 
distribution function for each. 


e The random variables described bv the characterizations are describing aggregated 
random events that are the result of many interactions within the model. Resulting 
random variables are viewed as convolutions of the many random variables 
associated with specific events generated within the model and present good 
characterizations of these aggregated events. 

Before examining the statistics it is desirable to develop an understanding of the nature 
of the distributions associated with each of these characterizations based upon real world 
events and actions. Not all of the distributions are intuitive nor will the fit of data be 
good. Selecting possible distribution functions a priori certainly provides greater 
confidence in distribution selection if the data fit of these distributions is acceptable. If 
the statistical results support intuitive distribution selection, then the conclusions will 
be stronger. 

Each of the characterizations are examined in depth in the following paragraphs. 
APL functions Were written to synthesize the essential data elements for each 
characterization. The function output, either in vector or matnx form, was then 
processed by GRAFSTAT. Due to the many possible GRAFSTAT options no specific 
reference to the GRAFSTAT function is stated. A general description of the analysis 
process is given and provides adequate guidance for reproduction. 

J. Frequency of shots while in battle 

There are a myriad of factors that impact on shot frequency. The most 
significant parameter impacting on shot frequency is the design of the fire control 
computer (FCC). Characteristics of the FCC on the AH-64 allow for three separate 
firing sequences: normal fire, rapid fire and mpple fire. Normal fire is when the gunner 
pulls the trigger and one mussile is launched. Rapid fire is selection of more than one 
missile and one trigger pull launches all missiles at eight second intervals. Ripple fire is 
selection of more than one missile and one trigger pull launches missiles at one second 
intervals. Given a mix of normal, rapid and ripple fire shots,the expected distribution 
will be skewed to the left due to the likelihood of shots being fired in the one to eight 
second launch interval. Normal fire sequencing takes much longer than eight seconds; 
generallv from 45 to 60 seconds. At time intervals near zero the likelihood of missile 
launch is zero. These characteristics, skewed to the left and passing through the origin, 
suggest the Gamma familv of distributions. The APL function 7VTSHOT listed in 
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Appendix E determined the intershot times from the data base. A total of 740 time 
intervals were extracted and analvzed in GRAFSTAT. Total shots numbered 742, 
yielding 741 possible intershot times; however one intershot time was negative (a 
possible postprocessor error) and discarded, resulting in 740 data points. Fitting the 
actual intershot times to a Gamma distribution results in the graph displayed in Figure 
1, along with the parameters of the distribution. 

The Kolomogorov-Smirnov (K-S) Test is used to measure the goodness of fit 
of the sample data points to the hypothesized distribution. The K-S Test uses a 
comparison of the empirical cumulative distribution function with that of the 
hypothesized cummulative distribution function. A bound of 0.95 was set on the K-S 
Test and is illustrated in the figure by the dotted lines. As seen in Figure | the fit of the 
empirical data to the hypothesized distribution is quite good. Appendix F lists the 
numerical values for the K-S Test as performed by GRAFSTAT and additional density 
function information. 


Listed below is the density function for the Gamma distribution. 


Pee | Te)” ° (1) 


15 


The parameters for this gamma distribution are: 

a = .6925$ 

B = 2.16 
A confidence interval of 95% for the Kolgomorov-Smirnov Test is shown 
on the graph and all sample data points fall within these bounds. 


INTERSHOT TIME DISTRIBUTION 
GAMMA CUMULATIVE DISTIBUTION FUNCTION 
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Figure 1. Intershot Distribution 


2. Number of shots fired as remote shots 
This information is needed to help develop the training time distribution 


between remote engagement training and autonomous engagement training. Remote 
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and autonomous firings were easily extracted from the data. The APL function 
REMOTE extracted the number of remote firings from the data matrix. In this 
particular scenario only the scout aircraft acted as remote designators for the AH-64’s. 
Since only two states exist, remote and autonomous, the binomial distribution accurately 
represents this distribution. Selection of either remote or autonomous designation 
should be a fixed probability given battle conditions do not change significantly. Battle 
conditions and parameters are assumed to be relatively constant throughout this short 
intense battle. Independence between shots is also assumed because of the independent 
nature of the helicopter target processes. Processing targets and performing other battle 
tasks are related but individual target processes are assumed to be independent. A total 
of 742 shots were taken during the 40 replications and 119 of those shots were fired for 
a remote target designator. Table 5 lists the statistic and a confidence interval for the 
distribution parameter p. A normal approximation of the binomial distribution was 
used to derive the confidence interval in Table 5. The 742 shots are a sufficiently large 
sample to justify using the normal approximation. The following equation was used to 


derive the confidence intervals in Table 5S. 
p+i.9ofF 2 and p-1.96/ (2) 


The binomial distribution 1s hsted below. 








Gipgg = Om = Onleo eet 
flx|n,p) = (3) 


0 otherwise 
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Table 5. PROBABILITY OF REMOTE DESIGNATION 


95% Confidence Interval [or 
BINOMIAL 1336 to .1863 










3. Number of targets in field of view (FOV) 

Knowing the expected number of likely targets eligible for prosecution helps the 
trainer build realistic target arravs. Presenting battle scenarios which best represent the 
actual battlefield conditions allows the crews to receive representative workload 
experience. The values for the points plotted in Figure 2 were derived from Blue AH-64 
target sensing events in all 40 replications. The APL function FOF’ created a vector 
listing the number of targets in the sensor's field of view each time the AH-64 conducted 
a target search routine. A total of 436 sensings were conducted by the AH-64’s. No 
distributions were derived intuitively, however the distribution must be discrete and the 
random variable will have relatively small integer values. All discrete distributions 
within GRAFSTAT were fitted to the data but none achieved acceptable fits. Figure 2 
is a plot of the cumulative distribution function of the data. The cumulative distribution 
function along with histogram derived information will be used to describe field of view 


Chiaractenzations 
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fieure 2 shows it 1s quite likely to sce few targets. 


mretiumber of tots <"2.0) = .5571. 


CUMULATIVE DISTRIBUTION FUNCTION 
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Figure 2. Number of TGTS in Field of View CDF 


4. Number of shots fired in 30 minutes of battle time 
This characterization is highly dependent upon the composition of the threat 
force and the suitability of the terrain for employment of the AH-64 aircraft. The 


selected scenario represents a high intensity battle with suitable terrain which vields a 


large number of missile engagements. Building training scenarios around characteristics 
of this scenario gives realistic and intense system employment. Total shots fired and 
total battle time for all AH-64 systems were the values extracted from the data base. 
Given that battle conditions do not change, the expected number of shots fired in any 
given time period should be normally distributed. Assumptions required to make this 
conclusion are that the battle intensity is homogeneous throughout the battle, 1.e., dead 
targets do not reduce the opportunity for engagements, and that each helicopter is 
fighting and firing missiles independent of the other helicopters. There are two tactical 
configurations for the AH-64’s in this scenario; paired with scout aircraft and fighting 
autonomously. The paired AH-64’s are fighting an independent battle but would not 
share the same environment with autonomous aircraft. Paired AH-64s fired primarily 
remotely designated targets. Onlv the attack configuration is addressed in Figures 3 and 
4. Figure 3 shows a normal cumulative distribution function fit to the shot data. The 
cutoff of data points at zero would cause problems if trying to use the normal 
distribution in a Monte Carlo simulation for the number of shots fired. The curve fit 
and mean values do provide insight into the total shots fired during the simulation. 
Figure 41s a fit of the same data to a gamma cumulative distribution function. The fit 
is acceptable but one data point, zero, was lost due to the characteristic of the gamma 
distribution that all x values be positive. In the case of paired AH-64 shot data, even 
more data points would have been lost using the gamma distribution because that data 
set contained numerous Zero points. In the case of autonomous firings only one data 
point was lost in 40 replications. Since the likelihood of zero shots 1s small, the loss of 
zero aS an output value when the gamma function is used is acceptable. The APL 
function OVERKILL developed a 4 X 40 matrix that lists the total shots fired in each 
of the 40 replications. Total shots were separated by tactical configuration, autonomous 


and paired, but individual aircraft firings were aggregated into the total shots for a 
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category. In the attack configuration, autonomous firings, three aircraft conducted the 
firing. As expected the intensity of finng in the two configurations was different. 
Appendix F lists the goodness of fit values for the K-S Test for the normal distributions 
as Well as the gamma distribution. Additional information about the sample data set 


and its fit to the hypothesized distributions is also given in Appendix F. 
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The parameters for this normal distribution are; 
= Ss 
ao = 7,946 
K-S bounds of .95 are displayed on the graph. 
Graph used for information and characterizations. 


AUTONOMOUS AH-64 SHOTS 
NORMAL CDF 


CUMULATIVE PROBABILITY 
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Figure 3. Number of Firings per Simulation Run (Normal) 


The parameters for this gamma distribution are; 
a= 2.4782 
B = 5.9078 
K-S bounds of .95 are displayed on the graph. 
Graph used for simulations and modelling purposes. 
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Figure 4. Number of Firings per Simulation Run (Gamma) 


5. Percent of hits from all shots 
Determining percent of shots that are hits aids in the establishment of standards 


for the gunners on live fire ranges and flight simulator exercises. The binomial 


distribution is used again because the outcome of a single shot is either a hit or a miss. 
Using total shot data and total number of hits data the p value for this binomial 
distribution was determined. The APL function OVERKILL derived the number of hits, 
given a shot, for each of the shots fired. Once again, the different tactical configurations 
were examined. The second row of the matrix, ATTACK lists the hits for the attack 
aircraft and the second row in the matrix SCOUT lists the hits for the paired AH-64s. 
(see Appendix G for matrix data). Table 6 gives the results of the analysis. The 
binomial distribution 1s approximated by the normal distribution due to the large sample 
size; 571 for autonomous and 119 for remote. Equation 2 was use to computesmae 


confidence intervals in Tables 6 and 7. 


Table 6. PROBABILITY OF uu GIVEN AUTONOMOUS SHOT 


Distribution %> Confidence Interval for p 


BINOMIAL 7259 to .7959 





Table 7. PROBABILITY OF erin) GIVEN REMOTE SHOT 


Distribution %> Confidence Interval for p 


BINOMIAL 5698 to .7402 





6. Number of multiple hits on targets 
This characterization is not directly related to a training task but does address 
One issue that concerns modelers and trainers alike; the realism with which the model's 
engagement modules portray the battlefield. Indirectly the trainer is concerned that the 
distributions derived are not flawed because the model 1s not simulating as expected. A 
specific concern to modelers is the number of targets that are hit more than once by the 


primary Weapon svstem. Expenditure of critical missile assets on multiple hits is not 
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wise, given the lethality of a single hit. Multiple hits do occur but are avoided unless a 
second shot 1s needed to insure destruction. Examination of how the model treats this 
real world occurrence gives greater confidence in the statistical results if treated 
realistically or allows the trainer to modify the results and train to preclude excessive 
multiple hits if treated unrealistically. No obvious distributions were postulated for this 
random variable a prion. The only conclusions made prior to distribution fitting were 
that occurrences of multiple hits were expected to be small and the distribution is 
discrete. The APL function OVERKILL determined the number of multiple hits in each 
replication of the simulation. Only the multiple hits of the attack configured aircraft are 
plotted. An excellent fit was achieved, verified visually as well as with the goodness of 
fit test given in Appendix F. Equation 4 is the density function for the negative binomial 
distribution. 


Get a Hee Oat 
fixir.p) = (4) 


@) otherwise. 
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The negative binomial distribution fits well with the following 


parameters; 
r = 5.000 
p = .25889 
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NEGATIVE BINOMIAL CDF 
Oo 
- —— 
© 
© 
e 
a 
zw 
iS Oo 
fo 
Lt 
= 
alee 
= 
Ss 
O 
a 
| 
0 2 4 6 


NUMBER OF OCCURANCES PER RUN 


Figure 5. Number of Multiple Hits (Attack) 


7. Expected survival time for single aircraft 
How well does the AH-64 survive on the high intensity battlefield? Survival 


time is examined to answer this question and gain a broader understanding of how 
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hazardous 1s the environment for the AH-64. A review of the detailed scenario and 
history file showed that the helicopter battle does not start at the beginning of the 
simulation. The simulation starts with the Blue forces in prepared defensive positions 
and the Red forces initiating the attack from their starting positions. Although random, 
the time it takes for the helicopter battle to start 1s approximately seven minutes after 
the simulation starts. 

Assume an exponentially distributed lifetime for the individual helicopter. The 


survival function of the individual helicopter is; 
F(t) = e r (5) 


This assumption 1s based on the following: 


e The helicopters are only a fraction of the friendly forces. So the situation, as seen 
from the enemv’s view, is a target-rich environment no matter how many 
helicopters have been killed. Stated in a different way, the eneniv’s concern is 
fighting the whole battle and not just engaging Blue helicopters. 


e Helicopters fight in cycles of engagement and coverage: this can be described as 
entering and exiting the battle many times. Each entrance 1s independent of the 
previous actions. It does not matter whether the helicopter enters for the Ist or for 
the 1th time. The assumed constant hazard rate, 4 . is a Weighted average of a low 
hazard rate in the phases of coverage and a higher hazard rate during the trmes of 
exposute. [he weights themselves are dictated by battle intensity, which over the 
battle 1s assumed to be constant. 


The model developed using the above assumption 1s one of eight items, Blue 
AH-64s, fighting forty independent replications. Since each replication 1s independent, 
the forty replications can be aggregated. The exponential model as postulated considers 
identically eight items tested for forty tests and three-hundred twenty items tested for 
one test. All replications end at the same time (25.0 min) but not all helicopter battles 
Start at the same time, as shown in APPENDIX G. The helicopter battle start times are 
quite close, therefore the minimum start time from the forty replications was used to 


simplify the calculation of the parameter, 1. The minimum start time was selected to 
preclude the occurrence of a negative life time should one replication produce an 
extremely early AHI-64 kill. The lifetimes of those AH-64s that died during the conduct 
of each simulation run was collected and the number of survivors was recorded. This 


tvpe of data 1s an example of Tvpe I censored data. 


a. Maximum Likelthood Estimator 
Let z, = battle start time to the death of the 1-th AH-64 (actual lifetimes) 
i, = battle start ume to the end of the simulation (censored lifetimes) 
The blue helicopter’s life times, 2s are independent and identically 
distributed with the above stated survival function, which has the cumulative 
distribution function F 


l—exp(-—/A) 120 


F(t, 4) = (6) 


0 otherwise 


and the density function f- 


Aexp(—/1r) r>0 


INEZ (7) 


0 otherwise 


n 


Lea Opener 
uncensored(d) censored(n — d) 


be the observed lifetimes of the helicopters. Then the MLE for the 1, 1s: 


d n 
L(A) = [} exe 24) [ T exo 29 (8) 


Taking the natural logarithm and differentiating In L( 4 ) with respect to / results in 
equation 9: 
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: d 
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Dat Dos 


| j=d+1 





= .027006 (9) 


The mean survival time for the AH-64 helicopters is the reciprocal of the parameter i; 


h = — = 37.028 (10) 
y 
b. Confidence Intervals 

To approximate confidence intervals for the estimated parameter y , Fishes 


information 1(4) was used to derive an estimate of the variance; 
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A 2 
y= er eer 105,500 (11) 
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The upper and lower 95% confidence bound for the parameter are approximately 






me — (ee004 (12) 


= 020971 & 4+(1.96) [— 
(2) 





The reciprocals of the confidence bounds for 1 were used to derive the confidence 
intervals of the estimated mean survival time. 

The APL function KJLLS was used to extract the number of aircraft killed 
and their death times. Another function LA.\/BDA was used to calculate the parameter, 
j. Table 8 lists the essential information of the exponential distribution that describes 


the expected lifetime of a single AH-64 on the battlefield. 


Table 8. EXPECTED LIFETIME INFORMATION 
Number ofa-¢ Deaths, 


ts caine, > 
oso Contidence Interval 












8. Range to targets when hit 

Range data is used extensively by the training developer in almost every facet 
of the training program. Knowing representative target ranges facilitates correct 
representation of targets on live fire ranges and in simulator visual displavs. Target 
range information significantly affects engagement time, missile time of flight and the 
pace of the copilot-gunner’s actions. All of these aspects of target range provides for 
better understanding of the nature of the target engagement process. For this analysis 
range data provides insights into the opportunity for simulataneous engagement of 
targets by both the copilot-gunner and the pilot. The copilot-gunner employs the 
Hellfire missile system while the primary weapon system for the pilot is the 30mm chain 
gun mounted under the aircraft. The simulation did not simulate the 30mm gun but 


review of engagement ranges provides insight into the possible opportunities for its use. 


a 


The type of targets engaged by the 30mm gun 1s restricted to lightly skinned armor 
vehicles. Range data should display some normal distribution characteristics given the 
fact that good gunners will try and conduct most of their firing at a specified optimal 
range. As the battle starts and wanes the ranges should be longer and shorter, 
respectively. The data show a normal tendency but is slightly skewed to the left. Figure 
7 displays a histogram of the ranges of lightly skinned target engagements by the Hellfire 
system. The 30mm gun could have been employed on targets that are less than 2500 
meters. The frequency of occurrence for ranges less than 2500 meters was very small. 

Equation | gives the distribution function for the gamma distribution. Using 
the cumulative distribution function for the hypothesized gamma distribution the 
probability of the range being less than 2500 meters is only 0.07593, calculated in 
GRAFSTAT. 
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Gamma distribution is outline by the curve overlaved on the histogram. 
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Figure 6. Range to TGTS (light) When Hit 


D. STEP 4 
The results of the analyses conducted in Step 3 are now related to specific training 


tasks. The context of this discussion will focus on the training development 
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requirements in preparation for operational testing of the AH-64. In the case of the 
AH-64, the training development work took place in 1980 with the operational test 
conducted Mav through Aug 1981. Little training development information was 
available in 1980. Development testing experience provided the sole source of 
information for operator tactical training development. 

In most cases it is rather easy to relate the results of the analvsis to specific training 
tasks. In all cases however the analvtical results do contribute to greater understanding 
of the conditions and standards of the critical training tasks. Training information 
gained in this process 1s significant in the fact that no other quantitative training analysis 
information 1s available. The characterizations and distributional information 1s the only 
available information for the conduct of training development. All critical information 
relating to each specific tasks 1s collected on the task analysis worksheet. Appendix H 
presents a sample Task Analvsis Worksheet for one AH-64 task. 

Each of the tasks specified in Step 2 will be discussed using the information gained 
from the analysis. An essential element of this step is the ability of the training 
developer to relate the derived analytical information to practical and useful information 
for the training program. The SMEs are specifically looking for insights into svstem 
operation and critical information concerning system employment. A discussion of 
insights and critical information is presented with each task description. Information 
and insights not related to a specific task are discussed in the last paragraph of this 
section and related to the overall AH-64 system training program. 

1. Task #6066: Search and Identify Targets 

The results of the T-MI analysis are counter intuitive to the assumptions and 
conclusions made in early training development for the AH-64. In 1980 the trainers and 
operators of the AH-64 expected target acquisitition would be easv and the capabilities 


of the AH-64 acquisitition system would present verv target rich displavs. The 
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distribution described in Figure 2 is not well defined but the histogram and cumulative 
distribution function do present valuable information. The probability of viewing two 
or fewer targets in a sensing sequence 1s 0.5571. Trainers and operators alike expected 
the number of targets presented in the displays to be much higher. During the 
operational test it was common to find operators waiting for a multi-target display 
before initiating the engagement sequence. The results of the analysis indicates that if 
you see one or two targets, do not wait for more target rich encounters. 

More important to the task of training is the knowledge that now the training 
developer has good reason to develop training programs that present target arrays with 
specific numbers of threat vehicles. The histogram 1s used to develop specific numbers 
of targets for simulator and live fire gunnery exercises. Cumulative distribution function 
characteristics are used in a Monte Carlo process to generate the number of targets 
viewed. This information is used directly in the development of the conditions for this 
task. This task is the initial step in the target engagement cycle and 1s a fundamental 
skill that significantly impacts on total system effectiveness. 

2. Task #6080: Engage Targets with Hellfire Missile 

This task is the most critical activity for the Weapon svstem. Most of the 
characterizations developed relate to this task. The distribution in Figure | describes the 
intershot time for the copilot-gunner. The mean of this distribution, 1.4 minutes, can 
be interpreted as measuring the firing intensity expected from a trained copilot-gunner. 
Training programs should develop operators to employ the Hellfire system at a simular 
pace. The number of shots fired in 30 minutes of battle is derived using equation 13. 
The battle time for each simulation displayed little variance, 0.3, with an average time 
of 16.576 minutes. Using battle time information and the mean number of shots from 
the hypothesized distribution in Figure 4 resulted in 9.06 shots per 30 minutes of battle 


time for each AH-64.. 
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30 min distribution mean 


Shots in 30 min = ———— eee 
average simulation ti. aircraft number 


— on (13) 
Thirty minutes was chosen as a maximum time for intensive training periods. When 
computing the expected number of shots for a single aircraft, the mean of the 
distribution was divided by the number of aircraft functioning in that 
mode’configuration. The assumption that battle conditions do not change for 30 
minutes allows extrapolation of the total shot count. 

Probability of hit information outlined in Tables 6 and 7 are additional 
standards that would be incorporated into gunnery exercises for the AH-64. Copilot 
gunners have a quantified standard to meet in their gunnery training. In 1981 there was 
no standard for timely sequencing of shots during training. During operational testing 
crews Were faced with multi-target opportunities and were unable to capitalize on the 
system's capability due to slow system engagement. There were no standards developed 
for the speed or frequency with which targets were to be engaged prior to the operational 
test. The intershot interval and total number of shots per aircraft for 30 minutes of 
battle give good practical insight into the intensity of Hellfire engagements. The single 
biggest Weakness of the operational test crew's was their inability to quickly and properly 
engage targets. 

3. Task #6082: Engage Multiple Tgts w/ Two Weapons Simultaneously 

Target range data give insights into possible simultaneous engagement 
Opportunities. From Figure 7 it can be seen that there are few light targets at ranges less 
than 2500 meters. This does present usable information to the training developer 
because quantitative rational is established to distribute training time between single 
Weapon engagement and simultaneous engagement. The sample data shows ten to 


eleven percent of lightly skinned targets are within range. This is even a smaller 
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percentage of the total target population. This training task needs to be addressed in the 
training program, but little accommodation and time should be expended. 

In preparing for the operational test, equal classroom time was allotted to all 
modes of fire. Simultaneous engagement was practiced and was expected during the 
conduct of the operational test. Results of the operational test, however, agree with the 
simulation results that none occurred unless specifically staged. 

4. Task #6404: Perform LOBL Autonomous Designation Engage Procedures 

Table 3 provides information to the training developer on the proportion of time 
the AH-64 was used in the autonomous firing mode. Training programs should reflect 
proportional time allotments for remote designation training and autonomous firing 
training. Proper allocation of training time to most likely modes of operation improves 
training effectiveness. Approximately 85% of the engagement training time should be 
devoted to autonomous engagements. 

Engagement training training time was not proportioned in this way for the 
Operational test training. Both engagement modes received equal emphasis although 
crews did prefer autonomous shots. 

5. Task #6405: Perform LOBL Remote Designation Engage Procedures 

Table 3 provides the same information asin Task #6404. Approximately 15% 
of engagement training should be devoted to remote engagements. This information 1s 
also useful for input into the OH-58D (the primary remote designator) training 
development effort. 

6. Additional Information 

Figure 5 presents information used to validate the output of the simulation. 
Confidence in the simulation’s ability to represent real world engagement sequencing 
gives greater confidence in the T-M] results. A characteristic of the Hellfire missile is 


that is produces a tremendous fire ball upon target hit. So tremendous is the fire and 


oo 


explosion that it would not be missed by combatants. The low incidence of multiple hits 
is realistic, given the nature of the weapon system. Multiple hits may occur when two 
missiles are launched at the same target within the time of flight for the shorter range 
missile. The low incidence of multiple hits, mean value 1.375, gives confidence in the 
model's ability to portray realistic engagements. 

Table 8 provides another observation of the general nature of the battlefield. 
The survivability data in Table 8, coupled with the intershot and total shot data, gives 
the aviation unit insights into the intensity of this particular battle scenario. This 
scenario 1s very intense and provides realistic information for the aviation unit to use in 
Its maintenance, rearming and support planning. This information validates the 
survivability of the AH-64 1n a high intensity battle. Crew confidence in the system's 


ability to fight and survive is an integral part of the training program. 
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IV. CONCLUSIONS 


A. METHODOLOGY 

The T-MI process used in this thesis is a straightforward application of standard 
data analysis techniques. The element that makes this application of data analysis 
techniques unique is its integration of training development requirements and analytical 
techniques. The characterization is the unique aspect of the methodology that integrates 
the training developer and the analyst. Traditionally, the trainer has looked to 
observable data for his training development information. Now, using surrogate 
observations via the simulation, he is able to view battlefield system emplovment and 
operation. 

The characterizations selected in the above analysis are not unique nor are the 
analytical techniques selected for describing the characterizations. The particular 
analytical method or technique is not important; what is critical 1s that the process 
provides the necessary information to the training developer. The methods and 
techniques addressed in this thesis demonstrate the objectives and mechanics of the 
methodology. 

This methodology 1s applicable to any weapon system that has high resolution 
simulation models available for analysis. Examples focused on emerging systems early 
in the acquisitition cycle but the methodology is equally applicable to fielded svstems 
needing training effectiveness analysis or training review. Systems that undergo 
significant modifications and justify new training development efforts also are prime 
candidates for T-MI analysis. The cost of training today often justifies review of current 
training programs and T-MI input into the review process is an additional tool to insure 


training programs address real world training requirements. 
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B. APPLICATIONS 

Application of the T-MI process covers every aspect of training. T-MI analysis 
produces information which in many cases 1s not available or is extremely expensive to 
acquire. 

1. Training Development for Operational Testing 

References throughout this thesis address the application of the T-MI process 

to the training development of an emerging system. In the case of the AH-64 the 
information produced would have been valuable to the operational test training 
program. The AH-64 was a system that differed significantly from then currently fielded 
systems. No suitable experience base existed to provide a basis for departure in the 
development of conditions and standards for AH-64 tasks. The T-MI process provides 
a distinct improvement in task information when SME and traditional data bases are 
lacking good task information. Another unique feature of T-MI information 1s that it 
is available as soon as the high resolution model has been verified and validated, which 
is several vears before the svstem 1s operational. 

2. Simulator and Procedural Trainer Scenario Design 

New major Weapon systems are requiring extensive use of simulators and 

procedural trainers. Training devices such as simulators and procedural trainers need 
representative, authentic and challenging displavs and scenarios. Using distributions 
developed from the T-MI process, the trainer can recreate key battle parameters and 
events in the training devices. Various target arrays and event sequencing can be 
accomplished through use of a Monte Carlo process using the probability distributions 
of the specific battlefield event. When randomness is based upon known distributions 
of battle parameters the training devices become more effective and truly complement 


the overall training program. 
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3. Training Program Review and Training Effectiveness Analysis 
Fielded systems experiencing training difficulties or systems undergoing 
programmed review can use the T-MI process to evaluate current training programs. 
In this particular application current training information is compared to the results of 
the T-MI analvsis. New insights into system operation may result in improved training 
techniques and system performance. 
4. Evaluation of High Resolution Simulation Modules 
Characterizing the behavior of the weapon system in the high resolution model 
allows the training community to verify system operation as represented in the 
simulation modules. A thorough understanding of the characteristic behavior of the 
weapon system in the simulation instills confidence in the simulation results. New 
weapon systerns are very dependent upon effective man-machine interface and an 
effective training program. Operator input must be quantifiable and within the limits 
of the target population. Characterizations of task events quantify operator 
requirements. Should operator requirements as simulated in a Weapon system 
simulation module exceed operator capabilities, then the simulation module must be 
modified to achieve realistic output. This application of the T-MI process takes place 
very early in the weapon system hfe cvcle during the simulation development process. 
5. Operator performance evaluation 
Applications of this methodology to training development are demonstrated, 
but an equally fruitful application 1s in the area of operator performance evaluation. 
The capability to use this process for real time performance evaluation 1s a result of 
ongoing activities within TRADOC. Training commands have and are developing 
instrumented range facilities for hardware testing and unit training evaluations. To date 
these facilities have focused on the evaluation and performance of hardware, tactics and 


doctrine. Hardware evaluation is best defined through operational testing. 
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Instrumented range facilities provide the capability to conduct quantitative operational 
testing. Operator performance during operational testing has become an issue in recent 
major operational tests. The T-MI methodology provides a quantitative 
characterization of system and operator behavior during test trials. Instrumented range 
data is similar to data obtainable from high resolution models. Using the T-MI process 
as if the trial data were simulation data gives the trainer the ability to characterize 
system and operator behavior relative to specific tasks. The focus of the T-MI process 
is on task characterizations and these characterizations give insights into operator and 
system performance. Figure | characterizes intershot time for the simulation; if used in 
performance evaluation it would characterize specific operator intershot times. 
Individual crews can be evaluated or aggregate crew performance can be assessed. This 


tvpe of analvsis sheds light on tasks needing additional training should system 


performance levels fall below expected standards. 


C. RECOMMENDATIONS 

The training community must make use of every available source for training 
development information. New weapon systems demand the utmost from the operator, 
and the trainer is faced with developing the training program early to insure procurement 
of needed facilities and devices. The T-MI process uses available information and 
personnel to extract valuable information from production high resolution simulations. 
It should be used earlv in the acquisition cycle to verify simulation output and then used 
continuously for training development. As updated simulations provide better 
representations of weapon systems and the battlefield continued application will help 
insure the training developer has the best available information. Training development 
challenges are tough, but through effective applications of quantitative analvsis the 


trainer can keep pace with technology and the threat. 
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APPENDIX A. SYSTEMS APPROACH TO TRAINING 


A. INTRODUCTION 

This appendix outlines the critical steps in the System Approach to Training (SAT) 
as they deal with the TM-I process. TRADOC Regulation 350-7 thoroughly describes 
and explains the use of SAT. The following paragraphs are taken directly from the 
regulation. 

The concept of a systems approach to training is based on an overall view of the 
training process. It 1s characterized by an orderly process for gathering and analyzing 
collective and individual performance requirements, and by the ability to respond to 
identified needs. The systems approach ensures the development of training which 
builds upon operational concepts, the force structure and the characteristics of new 


Weapons and systems designed for strategic missions and the evolving threat. 


B. THE PROCESS 
SAT uses five distinct processess in its development of training and training related 


products. The five processes are briefly outlined in this paragraph. 


e Evaluate. The term evaluate is used in the general judgmental sense of the 
continuous monitoring of a program or of the training function as a whole and 
involves both verification and validation. The process consists of internally 
evaluating the training program during each phase of its preparation while 
concurrently externally evaluating the overall training function. 


e Analyze. The process of analysis must include an extensive examination of the 
threat, doctrine, organization, geographical locations of units, resources 
constraints, tvpe of units, new svtems, and associated collective tasks. Examination 
of the collective tasks leads to identification of the individual tasks that must be 
performed to achieve combat readiness. 


¢ Design. The purpose of design is to ensure the subsequent systematic development 
of training programs and training support materials that enhance the overall 
efficiency and effectiveness of the total training system. The process is driven by 
the products of analysis and terminates in a blueprint of the training programs for 
subsequent development. 
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Develop. This process elaborates and supplements the products of design and 
results in training programs ready for implementation and training support 
materials ready for use. 


Implement. The implementation process involves the separate but related 
functions of preparing for and conducting training. 


C. FRONT END ANALYSIS (FEA) 


The FEA 1s the initial process for anv newly started training development effort. It 


comprises the bulk of the work effort within the analysis phase of the SAT methodology. 


The FEA employs a top down approach, beginning with the Battlefield 
Development Plan and the threat analysis, the unit table of organization (TOE), 
and the applicable doctrine publications to determine the unit missions. 


Unit missions are the primary tasks which the unit was orgainized and equipped to 
execute. The anlayst examines the TOE and the doctrinal publications to 
determine the answer to the question “Why does this unit exist?” The answer to 
that question is the type-unit mission list. 


Mission are further broken down into collective tasks. All tasks identified are 
categorized as critical. essential or other. Critical tasks are those tasks necessary 
to accomplish a unit's combat nussion. Essential tasks are those necessary to 
accomplish a unit’s peacetime or administrative support mission. 


The initial missions and collective tasks must be examined by subject matter 
experts (SME) and units in the ficld must be consulted. This is because the 
deternunation of WHAT must be done raises the issues of HOW it must be done, 
and HOW WELL it must be done. 


The results of the analvsts must be screened against the selection criteria in order 
to ensure that the selected mission, collective tasks, and leader and individual tasks 
are limited to those essential for accomplishing the unit’s true purpose or ensuring 
its Wartime success. 
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APPENDIX B. SIMULATION SCENARIO 


A. INTRODUCTION 

This scenario depicted an Armor Task Force defending against an attacking Soviet 
Tank Regiment. It was based on TRADOC Standard Scenario, Europe V, and an 
existing scenario which was used for studv at TRAC-WSMR. The combat situation was 


based on Critical Incident (CI) #4, of the Europe V Gaming Report. 


B. GENERAL SITUATION 

The scenario was set in V (US) Corp. A BLUE Armd Div and an ACR (+) 
defending forward while a BLUE In Div (Mech) (+) and an In Div (-) were in reserve. 
The BLUE Armd Div had completed its covering force fight and conducted a passage 
of hnes through MBA units. Covering forces in that sector were highly successful in 
stopping the advance of RED Guards Tank Div (GTD) Ist echelon regiments which 
then began assumming hasty defensive positions. In the BLUE ACR (+) sector, the 
regiment deployed two Cav squadrons and a RAS forward to act as covering forces. 
The threat, a Motorized Rifle Division (MRD), attacked with two motorized rifle 
regiments (MRR) in the first echelon. The BLUE covering forces fought back to 
prepared positions where they would defend in sector with a BLUE TF (M) in the north, 
a Cav squadron in the center, and a second Cav squadron in the south. The BLLE RAS 
conducted economy of force operations south of the second squadron while a BLUE TF 
eas recimental reserve. (See Europe V, volume 1, V (US) Corps Operational 
Scenario and Europe V, volume 1, V (US) Corps Gaming Report, Critical Incidents 1 
through 9), 

While the BLUE Armd Div was very successful in defeating the first echelon 
fwmenis Of the RED GID during the covering force fight, the BLUE ACR (+) was 
forced from successive positions more quickly than anticipated. This occurred because 
the first Cav squadron took the brunt of the attack by two regiments and was severely 
attrited. The regimental commander moved the regimental reserve, BLUE TF (A), 
forward to MBA defensive positions which the first Cav squadron was to occupy. The 
first Cav squadron passed through the BLUE TF (A) and reconstituted as the regimental! 
reserve. The threat, seeing that excellent progress was made, ordered the RED MRR 
to keep enemy forces fixed in the north and the second MRR to continue its advance 


south-southwest to expand the zone. The RED Tank Regiment (TR) was commutted 
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to complete the destruction of enemy forces to the division front to the extent that a 
second echelon division could subsequently conduct exploitation operations. 

This scenario portrays the RED TR attack. The BLUE TF (A) was committed in 
stages. During CI] #3, along with two troops of the second Cav squadron, one company 
of BLUE TF (A) was ordered into the first Cav squadron’s sector but arrived too late 
to stop the advancing first echelon regiments of the MRD. The regimental commander 
determined that his BLUE TF (A) would not be able to occupy the planned postion of 
the first Cav squadron because of the rapid rate of advance of the RED MRD. He 
ordered the BEWReI ere continue to defend in the north and the second Cav 
squadron to continue to defend in the south. The BLUE TF (A) occupied a blocking 
positon between and to the rear of the BLUE TF (M) and the second Cav squadron. 
The regiment permitted a shallow penetration to occur between BLUE TF (M) and 
second Cav squadron. BLUE TD (A) blocked the head of the penetration and BLUE 
Attack Helicopter Battalion (ATKHB) counterattacked on the flanks of the MRD 
penetration. 

The commander of BLUE TF (M) had a battle position prepared and gave BLUE 
Co (TF reserve) the operations order mission to occupy this battle position. This 
position defended the avenue of approach that threatened the southern flank of BLLE 
TF (M). The scout platoon of BLUE TF (M) screened the southern flank of the TF. 
The regimental commander ordered BLUE ATKHB to provide early warning to the IF 
(M) 1m the event that any enemy forces turned north toward the TF (M) sector. The 
second RED MRR 1m its attack southwest was not positioned where its forces could 
influence the southern attack helicopter company. 

The first echelon regiments of the RED MRD continued to open the shoulders of 
the penetration. The first RED MRR established a hastv defense in the north and fixed 
BLUE TF (M). The second RED MRR attacked south-southwest into the sector of the 
second BLUE Cav squadron to expand the zone. The RED IR, a second echelon 
regiment of RED MRD, was committed through the first MRR and second MRR to 
continue the attack. The BLUE TF (A) occupied a blocking position along the axis of 
advance of the RED TR. 


C. MODELING CONSIDERATIONS 
In order to transition from Europe V to this high resolution scenario, certain 
assumptions, modifications, and extrapolations from Europe V were necessary. The 


location of the battle was moved from its location in Europe V to take advantage of 


~4 


available digitized terrain. This scenario focused on the battle during the BLUE 
ATKHB counterattack. Only the forces that impacted on this portion of the battle were 
actually modeled. These forces consisted of the two northern companies of BLUE TF 
(A) (Tm C and Tm D) and the first echelon battalions of the RED TR (Ist, 2d and 3d 
tank battalion). Also modeled were the air defense, artillery and aviation assets that 
impacted on this portion of the battle. 

At the start of the war game, Tm C and Tm D of BLUE TF (A) occupied their battle 
positions, BLLE ATKHB moved into position to conduct their counterattack, and RED 
TR attacked with its first echelon tank battalions on three seperate routes. The lead 


elements of these tank battalions were 5-km away from Tm C at the start of the battle. 


D. COMPOSTION OF FORCES 

The BLUE force structure was the Army of Excellence, with equipment updated to 
1995 and updated air defense weapons (FOG-M, ADDATS) in accordance with the 
forward Area Air Defense Study. 

Die Svstems. 


Type Number 
M1A1 16 
Me 12 
ay 4 
M109A3 Moet NO. Of tubes ) 
M110A2 8 (No. of tubes) 
MLRS 1 (No. of tubes) 
M106A2 6 
OH-58D 3 
AH-64 5 
A-10 4 
ADATS 4 
FOG-M 2 
FAAO 1 
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REDS vstem=, 


Type Number 

FS! {23a 93 

BMP-F 42 

122mm SP How 24 (No. of tubes) 
152mm SP How 87 (No. of tubes) 
152mm SP Gun 22 (No. of tubes) 
VASILIK mortar 8 

HAVOC 

HAVOG-2 (air-to air) 4 

FROGFOOT 4 

FULCRUM 6 

SA-14 6 

Z5U-X 4 

SA7X-15 6 

SA-13 4 
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APPENDIX C. SAMPLE AH-64 TASK LIST 


A. MISSION TASK LIST 

The task hst shown in this appendix 1s only a listing of the mission tasks for the 
AH-64 as developed in 17 Dec 1982. There are a total of 213 training tasks for the 
AH-64 but only the 27 mission tasks are listed because they best relate to the use of the 


TM-I process. 


B. MISSION TASK LIST 
Task Number Task Description 


6025 Call for’Adjust Indirect Fire 
6064 Select and Occupy Firing Position 


6066 Search and Identify Targets 

6079 Select Appropriate Weapon Svstem 

6080 Engage Target with Hellfire Missile 

608 | Engage Target with 30mm Gun 

6082 Engage Targets w’ two Weapon Systems Simultaneouslv 

6083 Search, Acquire, Recognize, and Identify Targets 
with Day Television 

6084 Search. Acquire, Recognize. and Identify Targets 
with the Forward Looking Infra-Red 

6085 Search. Acquire. Recognize, and Identify Targets 
Maetne Wirect View Optics 

6086 Perform Target Tracking 


6087 Perform Target Handoff Procedures 
OUSS Operate Onboard Recording Svstem 
610] Engage Target with 2.27 FFAR 
6202 Operate the AWS-Puilot 

6205 Operate the AWS-Copiulot 

6302 Operate the ARCS-Pilot 

6303 Operate the ARCS-Copulot 


6304 Perform Initilization of ARCS Control Plan 

6402 Perform Weapons Arming Procedures 

6403 Perform PTWS Initialization Procedures 

6404 Perform LOBL Autonomous Designation Engagement 
Procedures 

6405 Perform LOBL Remote Designation Engagement 
ienOCeCUunes 

6406 Perform LOBL Ripple Fire Engagement Procedure 

6407 Perform LOAL Autonomous Designation Engagement 
Procedures 

6408 Perform LOAL Remote Designation Engagement 
Procedures 


6409 Perfron1 LOAL Ripple Fire Engagement Procedure 
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APPENDIX D. FUNCTION FOR CLEANING DATA 


The APL function SCRUB listed in this appendix was used to clean the data file 


obtained from the VAX postprocessor. The major tasks conducted within SCRUB 1s 


cleaning and organizing. The most critical function performed by SCRUB 1s 


construction of the matrices, SHOT1, SHOT2, SHOT3 and SHOT4. SCRUB must be 


run seperately for each of the four shot data sets. Workspace size limitations precluded 


aggregation. 

V SCRUB 
feaba| SHAPE1<pSHOT3 
[2] A<SHOT3=' ! 
[3] SPACES<(pA)p(a/[1] A) 
C4) COMSHAPE<((p SPACES )(11), (pSPACES )i2]-+/SPACES([isg 
[5] COMPACT<COMSHAPEp (,~SPACES )/(,SHOT3 ) 
[6] B<,COMPACT=! ! 
C7] COMPACT<«,COMPACT 
[8] COMPACTLB/1(x/pB)I]<'0! 
Es SHOT3<(pSHOT3 )p (, ~SPACES )VCOMPAGT 
[10] SHOT3(:42,50,51,53,79,87.0loet eo dle e's 
C11] SHOT3CCCSHOT30352]='F')/1i1¢(14+SHAPE1))352)<'1! 
[12] SHOT3CCCSHOT3(352]='ZL' )/1(1+SHAPE1))352)]<'2! 
[13] SHOT3CC(C(SHOT3(352]='I')/1(1+SHAPE1))352)<'3! 
C14] SHOT3CC(CSHOT3(352J]='M' )/1(1+SHAPE1))352]<'4! 
[15] 
C16) SHOT3C(CSHOT3L3521='C' )/7 pO eae ea 
C17] SHOT3CCCSHOT30352J]='H' )/1(1+SHAPE1))352]<'6! 
[18] SHOT3C((SHOT3(03107]='M' )/1(14SHAPE1 ))3107]<'1!'! 
[19] SHOT3CCCSHOT3[3107]='K' )/1(1+SHAPE1))3107]<'2'! 
C20] SHOT3CC(SHOT303107]='0'! )/1(1+SHAPE1))3107)]<'3! 
[21] SHOT3LCCSHOT3 12107 l=" el ho eee eee eee 
[22] SHOT3CCCSHOT3([3107J='D' )/1(14SHAPE1 ))3107)<'5! 
[23] SHOT3CCCSHOT3[3107]='H' )/1(1+SHAPE1))3:107]<'6! 
C24) SHOT3C((SHOT303108]4'0! )/1(1+SHAPE1 ))3108)<'0! 
[25] SHOT3CCC(SHOT3[310912'0' )/1(14SHAPE1 ))3109)]<'0! 
[26] SHOT3C((SHOT3[3110]24'0! )/1(1+4SHAPE1))3:110]<'0'! 
[C27] SHOT3CC((SHOT3[3111]4'0' )/1(1+SHAPE1))3111]<'0! 
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APPENDIX E. GENERAL APL FUNCTION LISTING 


The APL functions listed in this appendix were used to compile and extracted need 
data from the simulation history file. These functions worked directly with the large 
8812 X 19 matrix as well as with resultant vectors and matrices from other functions. 


The output values of these functions were the only informaion used in the statistical 


analysis. 
Peenrmoemar TIMe Ys 23: Y53Z2Z53S737I6:Y6:Y7:Y83 79; 
Pee 2e.20371537TI73;7TI8;3:TI9 

oe ZT<0 

e2) TI5<0 

3 J TI6<0 

C4j TI7<0 

eS ) TI8<0 

[6] TI9<0 

val A<1 


[8] L:Y<DATAL31J]=A 

[9] Z<«Y¥/(1] DATA 

f10] sf<Z[13;5] 

me Z27<Z2Z7,ST 

mae 5 (2Z036)=1)A0Z037)=105) 

C13) Y6<(Z2(3;6J=1)A(Z03;7]=106) 

mee Y7<(2Z036)=1)ACZ037)]=107) 

C15) Y8<(2036]=1)A(Z037]=108) 

mre Y9<(Z036)=1)A(Z037)]=109) 

C17) 25<Y5/[1] 

[18] 26<Y6/[1] Z 

mee 27<Y7/C1] Z 

[20] 2Z28<Y8/[1] Z 

meee 29<Y9/(1) Z 
( 
( 
( 
( 
( 


NS 


(22) 7I5<7TI5,((((149Z5)-1)9¥25035] )-S7) 

[23] 7r6é<TI6,((((14+9Z6 )-1)426035] )-ST) 

Mere FI7<TI7 , (CC (149Z27 )-1)4927035) )-ST) 

[25] 7I8<7TI8,((( (14928 )-1)4¥28035] )-ST) 

[26] 7I9<779,((((14p29)-1)4¥29035] )-ST7) 

[27] A<A+1 

[28] TIMES<(+/7TI5),(+/TI6), (+/7TI7),(+/7TI8),(¢+/7TI9) 
[29] 72Lx(A<u1) 


V DISPL 
fal) Pociyee Aso l=1 ACPATAL: 7 J=105 )A(DATAL3;10)=3) 
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[2] 
Esl 
C4) 
ESi 
C6] 
[e7ea) 
[8] 
[9] 
[08 
Tes liga 
Laue 
[13] 
fing 
15) 
C16] 
[aaa 
[18] 
[19] 
E2od 


Eola 
[23 
C3] 
C4] 
Loa 
C6] 
bea 


Ea 
[2a 
[33 
C4] 
ted) 
C6) 
oe 
C8] 


[9] 

C10) 
C11) 
[12] 


bola 


Y6<(DATAL:36)=1 )aCDATAL: 7 1=106 DN O272 210 =o 
Y7<(DATAL:6]=1)A (DATA S27 1=107) 400274 1- lence 
Y8<(DATAL36]=1 )A(DATAL37]=108 JA CDATA[L310]=3) 
Y9<(DATAL36]=1)A(DATAL:7]=109)A(DATA[L310]=3) 
Z5<Y5/[1] DATA 

Z6<Y6/[1] DATA 

Z7*+l7/ Pelee 

Z8<Y8/[1] DATA 

Z9<Y9/[1] DATA 

DIST<0 

A<1 


L:Y¥<«Z5C31J=A 


AGY/ Tie Zo 

GRID1<X(3;5,8,9) 

GRID2<(pGRID1 )o(( (1+ (pGRID1))+,GRID1),¢(, (GRIDIL(14+9GRID eee 
DIS<(C(GRID1I03 2) -GRID2(;2] )*2)+((CRID1133]-CRID2( 33) )*209 4a 
DIST]e]DI ST ,OLrsS 

A<A+1 

>Ex (4122 ) 


VY DiSRey Zeno 2 
Y<(DATAL:6]=1)ACDATA[310]=4 JACDATAL313]>43 )ACDATAL3s111= eo 
Z<Y¥/(l1] DATA 

X1<Z(038] 

Y1<Z(3;9) 

A2<Z(0314] 

Y7<Z ieee 

DIS<(((X1-X2 )*2)+((Y1-Y2)*2))*0.5 


VUROVs2 32-21 

Y< (DATAL36]=1)A(DATAL310]=1) 

Z<Y/C1)] DATA 

Ye (Z037)=101 DV (Z0 371 =102)vV(Z Es ieee 
Z1i«Y/C1] Z 

SCT<Z1(319] 

O<o SCL 

NUMSCT<+/Z1(0319) 

Ye (Z037)2105)V(Z0371=106 Vv (Zl 7 Jae 
- (Z-37-2108 )-(Z-37-=109) 

Z1iey/T1) Z 

ATK<Z1(3;19) 

D<pATK 

NUMATK<+/Z1(0319) 


V INTSHO?T 3 Y6:77° YS: YS: 26°27: 28325 
Y5<(DATA[L36J]=1 JA CDATAL:71=105 JA CO47 40-10 —s) 
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E2 | Y6<(DATA(C36)=1 )ACDATAC3;7J=106 )ACDATA[L310]=3) 
C3) ean Ase d=1 A CDATAL ;7 1=107 )ACDATAL:10)=3) 
i] Y8<(DATA(L 36] =1 )ACDATAL:7]=108 )ACDATAL310]=3) 
C5] Y9<(DATA[L:6J]=1 )ACDATAL37J=109 JACDATAEL310)]=3) 
C6] Z525/7 214)! DATA 

C7] Z6<Y6/(1ij DATA 

C8] Zi=v7/(1) DATA 

[9] Z8<Y8/C1] DATA 

oten Z29<Y9/{1J] DATA 

[11] ZIME<0 

fe2) A<i 

Mio £3: Y<Z5(L31J=A 

Ca4ujJ x<Y/C1J Z5 

C15jJ x<xX(35) 

ee) X1<19X%,25 

C17] Y<«x1-x 

C18] REC<(pY-1)4yY 

C19) ZIME<TIME ,REC 

C20] A<A+1 

[21] 25x(4124+1) 

[22] TIME5<TIME(CATIME] 


teheobos’Y; 2; 5L2ART Ss ¥ 


fe A<1 
27) KILLTI<10 
f3) ST<10 


Cu] L:YY<(DATAL31]=A) 

C5] Z<«YY/(1] DATA 

[6] START<+Z[13;5] 

C7] hea Z es 7 )=105VZ0;:7)]=106VZ(037)=107VZ0 37) =108vZ0;7]=109) 
C8] Mia ens osl=1)ACZ0319]=200000) 
[9] Y<YAY1 

mies. KITT<Y/(1) Z 

Mie KAILTI<KITIC; 5) 

mie RI LLITI<KILLTI ,KILTI 

[13] A<At+1 

C14] ST<S7,START 

mo +Lx(As41) 


V LAMBDA;D 
[1] D<pKILLTI | 
[2] SUMTI<+/(KILLTI-(MEAN STARTI)) 
[3] SUMTJ<+(200-pXILLTI)x((MEAN STOPTI)-(MEAN STARTI)) 
[4] LAM<D+(SUMTI+SUMTJ) 


WepO mre GG Ss YeZ:YYsZ2Z°X eAsZZZ2HK:OVRs XX: 


5] 


Lal 

[2a 

Gey 

C4) 

sa 

C6) 

C7) 

C8) 

C9) 

C10] 
Kalla 
wal 
[13] 
C14) 
pale 
C16] 
Ea lral 
[18] 
[19] 
i264 
Resid) 
R228 
[23] 
C24] 
P25) 
[26] 
yas, 
[28] 
[29] 
[30] 
Lea, 
[32] 
[33] 


Cala 
C2] 
esa 
Cul 
bos 
C6] 


Ee 
E24 
is] 
C4) 


OVER; SHOTS;HKSH;:OTH:Y1;SIZE 

OVER<10 

SHOTS<10 

HKSH<10 

OTH<10 

A<1 

Y<( (DATA[:19]=200000 )v (DATA[:19] =600000 )v (DATA[L:19]=500000)) 
Y14<(DATA[3:19]=100000 )v(DA7T[3:19]=300000 )v (DAT[L319]=400000) 
Y1B<DATAL[L3;6J=1 

Y<(YvY1A)aAY1B 

aY1<(DATA(L;:6J]=1 )aC(CDATA[3;7]=102 )v (DATA(L:7]=103 )v (DATAL- 7 
aAY1<(DATA[:6]=1 )aC (CDATAL: 7] =105)v QDATAI-71=104) 
aY1<(DATA[3:6J]=1)a((CDATA[:7]=106 )v (DATA[:7]=107 )v (DATAL: 7 ieee 
AY<YAY1 

Z<Y¥/(1)] DATA 
L:YY<Zl31ij=A 

SHOTS<SHOTS ,+/YY 

Z2eYY/(1) Z 

HK«(ZZ(03192=200000 )v(ZZ[319]=600000) 

HKSH<HKSH, (+/#K) 

ZELLeHK/([1) 22 

OTH+OTH, (+/(~H#K)) 

XSZ220715) 

SIZE<( px) 

XX*C(K1K )=1pX%)/X 

OVR<SIZE-pXxXXx 

OVER<OVER ,OVR 

A<A+1 

RESULT<((SHOTS,(O.5] HKSH),([1] OVER),(1iJ OTR 
aSCOUT<RESULT 

AATTACK<RESULT 

a O<OVER 

a O<OTH 

>+Lx(A<41 ) 


V RANGE: ¥:Z 

REMOTE 

Y<(DATAL36]=1)ACDATAL310]=4 )AC~YIMP ) 

Z<Y¥/C1] DATA 
TGTRNG<((0(Z0387-°-Z2Z0314) )*2 )4+¢(20393-20315] )*2))*0.5 
AVERNG<(+/7GTRNG )+pTGTRNG 

O<AVERNG 


V NUMRM<REMOTE3;Y13:Y2;Y3 

Y1<(DATA(36]=1 )ACDATAL:7J]=101 )a (DATA[; 10] =4 ) 
Y2<(DATA(:6]=1 JACDATAL:7]=102 )a (DATA 10n-—] 
Y3<(DATA(36]=1)ACDATA(L37]=103 JA CDATAL; 10] =4 ) 
YIMP<Y1ivY2vY3 


5] RMIMP<YIMP/(1] DATA 
Tle NUMRM<+/YIMP 


oe ee ee) ceZ 
fel J A<1 
2 | START I<10 
[3] STOPTI<10 
Cu) L3Y<«(DATAL31]=A)ACCDATAL36)]=1)v(DATAL36]=2)) 
[Sa Z<Y¥/(C1J] DATA 
C6] X<Z0135] 
ie?) STARTI<STABRTI ,X 
C8] X<Z((149Z)35) 
C9] STOPTI<STOPTI ,xX 
C10] A<A+1 
[11] 7>2x(As40) 


V NUMSH<TOTSHOT: Y3:Z23:YY¥3TI 
C1] A<1 
ie | FORSH<10 
C3) TISH<10 
moms ¥< (DATAL 311=A2 JACDATA([ 36] =1 )ACDATAL;3101=3) 
[5] Z¢Y/C1] DATA 
C6] eZ, LAs 2 | 
7 YY<+/Y 
fe] TOTSH<TOTSH , YY 
CQ] pon) hon ,7T 
[10] A<A+1 
C11J NUMSH<pTOTSH 
[12] 72x (As41) 
V 
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APPENDIX F.) FITTING PROBABILITY TABLES 


A. FIGURE 1-INTERSHOT TIME 
Fitting probability tables for the gamma distribution derived for Figure 1. 
ANALYSIS OF GAMMA DISTRIBUTION FIT 


DATA ey. epi! 

SELECTION > ALL 

X AXIS LABEL: MINUTES 

SAMPLE SIZE : 740 

CENSORING : NONE 

FREQUENCIES : 1 

EST. METHOD : MAXIMUM LIKELIHOOD 

CONF METHOD : ASYMPTOTIC NORMAL APPROXIMATION 


CONF. INTERVALS COVARIANCE MATRIX OF 
(95 PERCENT ) PARAMETER ESTIMATES 
PARAMETER ESTIMATE LOWER UPPER ALPHA BETA 
ALPHA OF692560°- Ur 05240 mone Zoe GO. 00093293 0. 0029S 
BETA Ze 1 S952 2.4248 OG] 0027505 0.018246 
SAMPLE FITTED GOODNESS OF FIT 
MEAN : 1.496 1. 496 
STD DEV : 1. 8485 1. 79u6 CHI-SQUARE : 113.278 
SKEWNESS: 7 0 2.4032 DEG FREED: 8 
KURTOSIS: SSNS 1s (ete) 8! SIGNIF 0 
KOLM-SMIRN : 0.2478 
PERCENTILES SAMPLE FITTED SIGNIF : 6. 8091E ge 
De O1353 0.024996 CRAMER-V M = 520790 
IMG Onises OV 069255 SIGNIF : <0 
Zoe Cals a3 O27 305 ANDER-DARL : 37.483 
SU: Cry aoe Omc65zil SIGNIF <7.20ll 
rier Zee oog 2.0534 
eO)r 3. 664 3.7645 KS, AD, AND CV SIGNIF. LEVELS NOT 
epi DED SS yee as EXACT WITH ESTIMATED PARAMETERS. 
CHI-SQUARE GOODNESS OF FIT TABLE 
LOWER UPPER OBS EXP O-E ((O-E)*2)°E 
SINE. 0. 9065 418 57 33ae S186 SS) Bo) 1Ue6: 
029065 ieS3 54 148.95 94. 946 60.524 
Lice BS) 227 9 Sle S O2.g07 505 OhSNS: SUZ» 
Ze oS S626 5)S) 49.062 oh 2 SIS 2. OG 
5,026 4.5325 24 Ze a2. Demo. 2 L,. 13st 
5325 Deo 9 14 18.409 4.4089 ICS Se) 
5.439 6.3455 8 11.488 3.4884 052 
6.3455 eae 10 7, 2235 2 Oo 0G 72 


tee) 2 Salo o> Wi 4.5674 on 
ig alse bs. oeo7 LS 8 4.7489 on 
mag715 +INF. 1 3.3034 ae 
TOTAL 740 740 


4326 9.0594 

2511 2h (a faa ie 

3034 1. 6061 
IES rae) 


B. FIGURE 3-TOTAL SHOTS (NORMAL DISTRIBUTION) 


Fitting probability tables for the normal] distribution for oral shor data. 


ANALYSIS OF NORMAL DISTRIBUTION FIT 


DATA ATTACK[ 13] 
SELECTION ALL 
X AXIS LABEL: NUMBER OF ENGAGEMENTS 
SAMPLE SIZE: 40 
CENSORING : NONE 
FREQUENCIES : 1 
EST. METHOD : MAXIMUM LIKELIHOOD 
CONF METHOD : EXACT 
CONF. INTERVALS 


(SSS EERCENT ) 


PARAMETER ESTIMATE LOWER UPPER MU SIGMA 
MU 14. 275 7 Od 16. 849 ew ci7 Wu, 
SIGMA fee 0/ bees l03S6 0 Ui? 627 
SAMPLE PItTeD GOODNESS OF FIT 
MEAN 14.275 14.275 
SiveDEV : 8.0479 7.9467 CHI-SQUARE : PeulGoe 
SKEWNESS: 0.036654 0 DEG FREED: 4 
RORTOSIS: 1.9458 3 SIGNIF O!. 70577 
ROMS SMIRNG se O LO9] 7 
PERCENTILES SAMPLE na TED SIGNIF ec 2/09 
oy 1a ie 1 1 CRANER-V te: 0.09102 
10; Ded 4.0895 SIGNIF : ame Bs 
ZS: 7 Oa 175 ANDER-DARL : 0.5194 
0) il) 14. 275 SIGNIF cared le, 
iS: et ILS) ae): 
ASE 24 24.46 KS, AD, AND CV SIGNIF. LEVELS NOT 
aD : 47) 2) oe EXACT WITH ESTIMATED PARAMETERS. 
CHI-SQUARE GOODNESS OF FIT TABLE 
EGWER UPPER OBS EXP o=h ((O-E)*2)°E 
PENT. 4.2857 5 4.1748 0. 82524 eam r, \1 
4.2857 8.5714 fi By Paley Fi 17163 Dov ock 1 
8.5714 1 2eeGo7 5 7.7094 2.7094 Geo el9E 1 
no) / 17.143 i 8.4685 1.4685 Zo 405E 1 
17.143 21.429 8 7.0034 0), SSRs ys) 6 otook | 
21.429 25.714 5 4.3602 0.63984 S) 3 SKe}o) 5 oar 


COVARIANCE MATRIX OF 
PARAMETER ESTIMATES 


> 


2 See a TNE. > S000 0.00012549 ayy 4 AS) 
TOTAL 40 40 2. FOS2ZE0 


C. FIGURE 4-TOTAL SHOTS (GAMMA DISTRIBUTION) 
Fitting probability information and distributional information for the fit of zozal shor 


data to the gamma distribution. 
ANALYSIS OF GAMMA DISTRIBUTION FIT 


DATA > XX 

SELECTION > ALL 

X AXIS LABEL: NUMBER OF SHOTS 
SANE ER SiZh yee 

CENSORING : NONE 

PREQUENGUES >a 

EST. METHOD : MAXIMUM LIKELIHOOD 


CONF METHOD : ASYMPTOTIC NORMAL APPROXIMATION 


CONF. INTERVALS COVARIANCE MATRIX OF 


(95 SEE RCEN) PARAMETER ESTIMATES 
PARAMETER ESTIMATE LOWER UPPER ALPHA BETA 
ALPHA 2.4782 1.4435 Seeors OF 2764 0.66414 
BETA 5.9075 3.1743 8.6414 0.66414 1.9444 
SAMPLE FITTED GOODNESS OF FIT 
MEAN : 14. 641 14. 641 
SDI: 7.8085 9. 3004 CHS QUARKS: One o ee 
SKEWNESS: 0. 020886 1205 DEGST REED: 4 
KURTOSIS: 1. 9437 5.4211 SIGNIF > 0, 055162 
KOLM-SMIRN : 0.16154 
FERCENTIEES SAMPLE PeieD SIGNIF i 0. 20065 
5: gu Sree og CRAMER=V M >. 0.1693 
io: 4 4.6831 SlGNIt : ae a 
2D; 7 720019 ANDER-DARL : 0.94996 
a0; US ZZ 6 SIGNIF ; Zoe lls 
7D! Zl 19.414 
oO: 25 rarer KS, AD, AND CV SIGNIF. LEVELS NOT 
Op: ZG S250 EXACT WITH ESTIMATED PARAMETERS. 


CHI-SQUARE GOODNESS OF FIT TABLE 


LOWER UPPER OBS EXP O-E ((O-E)*2)%E 
- INF. 4.1429 4 3.0618 g0:93816 0028745 
4.1429 8.2857 7 7.6744 0.67436 0.059257 
8.2857 12.429 5 8.219 ge 19 1. 2607 
12.429 Lie SIA 4 647545 ee 545 wise 
16.571 AO, Fes 8 4.8794 3.1206 1.9958 
AO Te 24. 857 7 3.2655 3.7345 Le OM 
24. 857 +INF. 4 5. 1454 ao 0.25498 
TOTAL 39 39 9. 2522 
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D. FIGURE 5-MULTIPLE HITS 


ANALYSIS OF NEGATIVE BINOMIAL DISTRIBUTION FIT 


DATA 7 AL LACK Earal 


SELECTION >: ALL 


X AXIS LABEL: NUMBER OF OCCURANCES PER RUN 


SAMPLE SIZE : 40 


CENSORING : NGNE 
BEE@UENCIES : 1 
PARAM SPEC : N= 5 


PARAMETER ESTIMATE 


N 5) 

P 0. 25889 
SAMPLE 

MEAN : eos > 

ean DEV : 1.3144 


SKEWNESS: Oa o017 
KURTOSIS: moO 5SS 


PERGENIILES SAMPLE 
Se 
ILO 
Eo: 
50: 
Task 
90: 
315) 


FEOF OOO 


AND P = Q. 25889 


COVARIANCE MATRIX OF 
PARAMETER ESTIMATES 
NOT AVAILABLE 


Parl ED GOODNESS OF FIT 
P2945 
ieee OD CHI-SQUARE : Oe LO7 77 
Leo PEGS EREED: ) 
4.8137 SIGNIF poo 9069 
FITTED 
0 
0 
0 
i 
Z 
2) 
4 


CHI-SQUARE GOODNESS OF FIT TABLE 


BOWER UPPER OBS 
SINT. Slee) 2 

0.5 eas) 13 
i> 25> 8 
Zi TUNE: 7 

TOTAL 40 


We 
13 
8 
6 
40 


EXP O-E ((O-E)*2)%E 
651 0. 65089 3.3488E 2 
. 008 0.0082408 5.2206E 6 
o254 0.025412 8.0463E 5 
.3155 0.68454 7.4199E 2 
leOn 7 Deane 


FE. FIGURE 6-RANGE OF TARGET WHEN HIT 


ANALYSIS OF GAMMA DISTRIBUTION FIT 


DATA obs 
SELECTION : ALL 


X AXIS LABEL: RANGE (M) 
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SAMPLE SIZE : 
CENSORING 
FREQUENCIES : 
ESl, MEGHOD 
CONF METHOD: : 


Ba 

NONE 

1 

MAXIMUM LIKELIHOOD 

ASYMPTOTIC NORMAL APPROXIMATION 


CONF. INTERVALS COVARIANCE MATRIX OF 
(95 PERGEND) PARAMETER ESTIMATES 
TIMATE LOWER UPPER ALPHA BETA 
62597 51. 7345251950 6.7747 hfe: Jy, 
7.644 75.421 99RBG7 se Vs, 38/8710 
AMPLE FITTED GOODNESS OF FIT 
ZZ oe 3220.5 
53627 2 531.94 CHI-SQUARE : 46. 803 
O. SSNS O232953 DEG FREED: i 
4.0588 Selo 2g SIGNIF 6.0978E 8 
KOLM-SMIRNGs) C1022 
SAMPLE PET TEE SIGNIF ; 0. O0OOTSGs2 
2407. 2 2406. 1 CRAMER-V M: 0.74379 
265009 250625 SIGNIF O25 
290 Omg Zo 55a ANDER-DARL: 4.541 
5130 3199.4 SIGNIF Soil 
ESTES TE: B>O0:2 
SIONS 5926 KS, AD, AND CV SIGNIF. LEVELS NOT 
4510 4150.3 EXACT WITH ESTIMATED PARAMETERS. 


CHI-SQUARE GOODNESS OF FIT TABLE 


PARAMETER ES 
ALPHA 5 
BETA 8 
5 

MEAN : 3 
SLD DEV: 
SKEWNESS: 
KURTOSIS: 
PERCENTILES 

5° 

LO: 

25: 

50: 

Fins 2 

910: 

5) Sy: 
LOWER UPPER 
- INF. 2329. 
2329-6 2620: 
2620. 8 Zoe 
2912 3203: 
S20ee7 3494, 
Coe 37.65. 
378526 4076. 
4076. 8 4368 
4368 4659. 
4659.1 seth ie 
TOTAL 


O-E ((O-E)*2)°E 
5937 0.95004 
6126 2.6696 
0771 0.017599 
502 7.6065 
3715 0.67394 
1713 1.1478 
423 7 lily, 
7937 4.8962 
699 Yue aut 

.055341 0.0010024 

46. 803 
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APPENDIX G. SAMPLE DATA MATRICES AND VECTORS 


A. HISTORY FILE DATA MATRIX 


The matrix listed in this appendix is a sample from the larger data matrix used in the 


TM-I analysis. The data matrix is 8812 X 19 and the sample is only 5 X 19. Each 


column element provides event information. Below the matrix display is a key used to 


interpret the data matrix. 


SHOT3 
new 1== >All iOS 7 23 7 2 i 
107 8300 8700 4 80 0 
1 8264 5649 peSOK 7) BU>Z 85.5 
200000 
rem 2-- on AR oo Zo 10 2803 IL 
O7 8300 8700 3 80 Ole hess, 
20 8378 5244 178.7 3457 0 
1 
oye - = eal 1 eke WS 8 B22 1 
107 8300 8700 4 80 0 
20 8364 5247 MPs} 22) 3453 74.3 
200000 
row 4-- al i657 2s a 4136 1 
107 8300 8700 3 80 Oe 
zl 8396 5239 178.4 3462 0 
IL 
ney > -— Sil 0.07 23 10 4788 1 
107 8300 8700 4 80 0 
21 6375 5244 17896 3456 74, 2 
600000 
Matrix Key 


col l--replication number 


col 2--engagement start time 

col 3--engagement identification number 

col 4--shot identification number within engagement 
col 5--event type time 

col 6--side identification; 1-BLUE, 2-RED 

col 7--player identification 

col 8--player easting grid 

col 9--player northing grid 
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col 10--event type; l-sensor cycle, 2-select weapon, 3-fire, 
4-round impact, 5-call paired a/c, 6-a/c mission order 

col ll--weapon identification number of firer 

col 12--event duration minutes 

col 13--target identification 

col 14--target easting grid 

col 15--target norehiaceeied 

col 16--gun target line azimuth 

col 17--range to target 

col 18--number of tgts in search sector, firer speed, aspect 
angle of tgt 

col 19--impact results; 100000-miss, 200000-kill, 300000-other, 
400000-false tgt, 500000-dead tgt, 600000-hit, tagret 
speed, number of tgts in FOV 


B. AH-64 DEATH TIMES 

The times for the deaths of the 77 AH-64 kills are listed. These kills are the 
aggregated number of kills for all 40 replications of the simulation. These data were used 
to compute the expected lifetime information for the AH-64. Battle time for each 


replication was computed by subtracting STARTI from STOPTI. 


STARTI 
8.0166 
7.6936 7.6785 7.6875 9.095 6.0393 7.738 9. I45557 ee) 77624 See le 
7.6892 7.6736 7.6462 7.6646 7.675 7.7097 7.6924 5756731 7.661997 7a 
7.7195 7.7434 7.6963 7.6543 7.7178 7.7424 7.6709) 6.0227 7.73542 Oe 
7.6614 7.771 9.113 7.667 7.57129 7.6531) 7. 647792 meeee a. 7074 

STOPTI 
24.2876 


23.2463 24.4722 23.7056 24.9663 23,9619 23. 7 192 2ae 549 2450 be 

24.3074 24.4741 24.6443 24.8865 24.9885 24.0999 24.498 24.9912 24. 7844 
23.1594 24.6379 24.2947 24.186 24.8179 24.6921 24.8574 23.51 24.5186 
24.9209 24.6055 24.4514 24.9045 24.5859 24.9795 23.7903 23.9724 24.8252 


Kio 
Pore oy LE, 
8.0972 22.9758 8.696 22.7258 8.0522 23m7056 2062528228002 ee ee 
8.1106 21.5859 22.7515 19.4666 8.5289 23. 53520223 eee 
8.0447 20.3591 22.6743 8.0891 2253477 23. 0Gs N24 eco ome Osc my aU le 
22.3455 8.3379 18.9148 19.8621 24.0437 8.0354 9207300513. 9373) 225Gere 
8.8005 22.0073 8.269 22.0894 23. 1592 8 0357 cee ee 
19.4529 20.1824 24. 2051 624536923. 7117 24.5557 855207 eee ae lee 
23.0969 19.9436 21.0972 22. 741 21]. 7332) 24. 0s 7 ee 
8.0476 8.1267 8.2979 21.0327 20. 8997 225 72a Cm Gn eee i ee 
8.075 21.0371 22.8516 24 5208 
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C. SHOT EFFECTIVNESS DATA 

The two matrices, ATTACK and PAIRED, list the results of the firing data within 
the history file. The ATTACK matrix lists data for only the AH-64s operating 
autonomously. The second matrix SCOUT contains the same information but only for 
those AH-64 aircraft operating in the paired mode with scout aircraft. 

ATTACK 

Mioey > O13 25 6 30 21 8 5 
fmemeeeety 17 22°52 7 4&4 12 23 1:14 20 «11 21 
foie 21 19 15 19 15 28 3 6 9 18 26 
feeeeee 0 11 20 224 16 6 4 14 


feels) 6 207011018 «41 «12 «15~«€68 («15 
foerer 16 17 14 1318 8 23 2 4 8 10 21 


meer Oo 2 4 0 4 3 0 1 il 
meee OCU TCUCOODCUCND CSC dd COCOODlhcd1l CUO? ld 
fees Ss 2 | 40411 20 2 
fmmeoes 0 25 4 6 5 2 «1 «6 
eee? O OU S$ 2 5 0 2 5 3 6 
feo. |6mS|6UCODUCUlCUdNlCUdFlhUcSKlUdlhCUc kU C8 CSS 
PATRED 
Meeerovoooo0o02zo400400001 
Pm 0 S3S0038333204012040 
mmo O00 0020300300000 
moO 1003g22103011040 
Mem o00 0000000000000 0 
moro O00 000000000000 0 
femeeo oOO0O00 1l010O0TO000! 
mmo O22 0001110100100 0 


¢ row-l, total shots fired during each of the 40 replications of the simulation 
e row-2, total hits achieved by the AH-64 in each of the 40 replications 


¢ row-3, number of occurances within a single replication that a target was hit more 
than once by a Hellfire missile. 


¢ row-4, total number of non-hit missiles; misses, false targets, break laser lock, etc. 


61 


APPENDIX H. SAMPLE COLLECTIVE TASK ANALYSIS WORKSHEET 


A. COLLECTIVE TASK ANALYSIS WORKSHEET 
The task analysis worksheet is the primary instrument used by the training developer 
to catalog and collect critical information on task performance. The worksheet 


presented is a sample AH-64 task analvsis worksheet. 


B. SAMPLE WORKSHEET 
COLLECTIVE TASK ANALYSIS WORKSHEET 


ORIGINATOR: DOTD, USAAVNC 
DATE: Wan 1950 


CRITICAL MISSION(S): Perform fire support for manuever elements 
Perform anti-armor fire support 
Perform fire and maneuver 


ELEMENT: AH-64, single aircraft 


COLLECTIVE TASK: Engage targets with Flellfire Modular Missile 
System (HMMS) 


CONDITIONS: The AH-64 has occupied its firing position with 
suitable fields of fire. 


Target arrays are composed of threat tanks, BPMs 
air defense assets in quantaties described bv the 
cumulative distribution function presented in Figure 
? 


foe 


Battle intensity is such that total shots fired 
in a 30 minute battle will be greater than or equal 
to nine shots per aircraft. 


STANDARDS: Rate of fire is dependent upon target acquisition, 
however, copilot-gunner must be able to achieve 
a mean intershot time of 1.4 minutes in a high 
intensity battle. 


Gunners must be able to achieve 0.75 probabilitvof 
hit for all shots in the autonomous mode and a 0.65 
hit probabilitv in the remote engagement mode. 


Crew flight tactics and maneuvers must reflect 


proper emplovinent techniques that result in a mean 
survival time of 37 minutes. 


SUBTASKS AND STANDARDS: (Lists all pilot and copilot-gunner 
cockpit and weapon system tasks to conduct a 
missile firing in all modes. Onlv 
reference 1s made to these tasks and standard 
because they are numerous.) 


EXISTING INDIVIDUAL AND LEADER TASKS: (Lists the 
platoon and section leader, pilot and 
copilot-gunner tasks currently taught that are 
directly related to missile firing and target 
engagement, again too numerous for 
listing here.) 


NEEDED INDIVIDUAL AND LEADER TASKS: (New tasks 
that need development for the platoon and section 
leader as well as the crewmembers. Tasks not 
listed due to volume.) 
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